Networked computer systems are becoming increasingly popular as they permit different computers to share information. Very often in a network, some systems and their resources are not as highly utilized as other systems on the network, and so there has been interest in developing distributed applications to better utilize the available resources within a network.
Designing distributed applications such as, for example, memory systems, file systems and lock managers is difficult for a variety of reasons. These, for example, include issues of consistency of the distributed state, the reliability and availability of data in the face of node, link or other component failure, and scalability. A node is generally a device that is connected as part of a computer network. Not to the exclusion of other devices, as used herein a node is generally understood and appreciated to be a computer.
With respect to memory, memory for a computer system is technically any form of electronic, magnetic or optical storage; however it is generally divided up into different categories based in part upon speed and functionality. One category is mass storage—typically permanent, non-volatile memory stores which are generally understood to be cheap, slow, and large capacity devices such as hard drives, tape drives, optical media and other mass storage devices. The primary object of mass storage devices is to store an application or data until it is required for execution. With low cost and a non-volatile nature, mass storage devices typically provide a computer system with memory stores ranging to the hundreds of gigabytes. These mass storage devices operate with access times generally in excess of one millisecond.
A second general category is application memory and it is intended to permit quick access for processing, and is connected by a memory bus directly to the computer's processor. In contrast to the relatively slow storage memory, the main memory is generally comprised of fast, expensive volatile random access memory (RAM) with access times generally less than one hundred nanoseconds. Due to high cost and the volatile nature requiring continuous power, main memory RAM is typically provided in a size range from a few tens of megabytes to several gigabytes.
With distributed systems, the distinction between mass storage memory and main memory can be, and often is, blurred in the effort to capitalize on unused resources in the network environment. A number of different systems and methods are known and currently implemented in a variety of forms, in efforts to provide distributed systems.
For example, databases systems are well known and widely employed as a means to centralize and coordinate the storage of information. However, databases are not commonly used for applications that would traditionally be executed in main memory as they are slow and access commands are orders of magnitude less efficient. As their primary function is data storage, databases provide broad sets of functions, which in turn impose additional overhead for operation and interaction. As an example case in point, relational databases require data to be stored in tables and then manipulated by relational algebra operators. This is very different from traditional interactions with memory in which the application processing is actually occurring, and as such, database systems are not truly applicable for use as application memory.
Another example is a network file system, which again is primarily oriented towards providing network storage systems. A network file system typically involves a centralized server to control all access transactions between application nodes and the data stored in the file system. With a single point of access, a network file system can and often does succumb to issues of overload and forced latency as the centralized server attempts to respond to more requests than it can manage at any one point. The centralized server is also a single point of failure which, upon failing, renders the entire network file system unusable.
A distributed file system provides better scalability and access control than a network file system, but again is primarily focused upon providing the storage and retrieval of data from mass storage devices for use in application memory. Disk arrays and storage area networks are also increasing in their frequency of use within networks, but again their principle function is as mass storage, not application memory. In addition, there is typically little if any support for simultaneous write operations performed by multiple nodes to the same data.
Shared memory systems provide an abstraction of reading and writing to a location in a shared memory environment, e.g., a shared application memory environment. Specifically, multiple processors utilize a shared memory environment as a way to communicate and maintain synchronization. Typically, such a system is limited to a small set of nodes and enjoys short physical distances between the nodes.
Distributed shared memory systems are yet another example. Typically, in a distributed shared memory system the nodes communicate with messages. The nodes are not differentiated such that a node being used for the processing of an application may also provide memory resources to other nodes in the network. This has the negative impact that if the processor is busy processing an application task, its shared memory access time will slow until the processor becomes free from the application task.
In addition, in a shared memory environment it is often extremely important to control write operations in such a way that other nodes do not inadvertently receive partial data or data believed to be current when in fact it is being rewritten. Methods to impose such control and conditionality are frequently quite complex, thus imposing additional overhead resources and latency issues upon the system.
Hence, there is a need for a networked transactional memory system and method for directing transactions in such a distributed system that overcomes one or more of the drawbacks identified above.